MEDB 5502, Module 13, Bayesian Statistics

Topics to be covered

What you will learn
- Classical statistics
- Bayesian statistics
- Prior distribution
- Likelihood
- Posterior distribution
- Applications of Bayesian statistics

Classical statistics, 1 of 3

\(H_0:\ \pi_1=\pi_2\)
\(H_1:\ \pi_1 \ne \pi_2\)
- \(\pi_1\), \(pi_2\) are fixed constants

Classical statistics, 2 of 3

\(T=\frac{\hat p_1-\hat p_2}{s.e.}\)
p-value = \(P[Z > T]\)
- probability of sample results or more extreme
- p-value is not \(P[H_0]\)

Classical statistics, 3 of 3

95% confidence interval
- \(\hat p_1-\hat p_2 \pm Z_{\alpha/2}s.e.\)
- Range of plausible values
Probability statements not possible.

Break #1

What you have learned
- Classical statistics
What’s coming next
- Bayesian statistics

A simple example of Bayesian data analysis.

ECMO study
Treatment versus control, mortality endpoint
- Treatment: 28 of 29 babies survived
- Control: 6 of 10 babies survived
- Source: Jim Albert in the Journal of Statistics Education (1995, vol. 3 no. 3).

Bayesian data analysis seems hard, and it is. Even for me, I struggle with understanding Bayesian data analysis. In fairness, I must admit that much of my discomfort is just lack of experience with Bayesian methods. In fact, I’ve found that in some ways, Bayesian data analysis is simpler than classical data analysis. You, too, can understand Bayesian data analysis, even if you’ll never be an expert at it. There’s a wonderful example of Bayesian data analysis at work that is simple and fun. It’s taken directly from an article by Jim Albert in the Journal of Statistics Education (1995, vol. 3 no. 3) which is available on the web at www.amstat.org/publications/jse/v3n3/albert.html.

I want to use his second example, involving a comparison of ECMO to conventional therapy in the treatment of babies with severe respiratory failure. In this study, 28 of 29 babies assigned to ECMO survived and 6 of 10 babies assigned to conventional therapy survived. Refer to the Albert article for the source of the original data. Before I show how Jim Albert tackled a Bayesian analysis of this data, let me review the general paradigm of Bayesian data analysis.

Wikipedia introduction

P(H|E) = P(E|H) P(H) / P(E)
- H = hypothesis
- E = evidence
- P(H) = prior
- P(E|H) = likelihood
- P(H|E) = posterior

Prior distribution

Degree of belief
- Based on previous studies
- Subjective opinion (!?!)
Examples of subjective opinions
- Simpler is better
- Be cautious about subgroup analysis
- Biological mechanism adds evidence
Flat or non-informative prior

The first step is to specify P(H), which is called the prior probability. Specifying the prior probability is probably the one aspect of Bayesian data analysis that causes the most controversy. The prior probability represents the degree of belief that you have in a particular hypothesis prior to collection of your data. The prior distribution can incorporate data from previous related studies or it can incorporate subjective impressions of the researcher. What!?! you’re saying right now. Aren’t statistics supposed to remove the need for subjective opinions? There is a lot that can be written about this, but I would just like to note a few things.

First, it is impossible to totally remove subjective opinion from a data analysis. You can’t do research without adopting some informal rules. These rules may be reasonable, they may be supported to some extent by empirical data, but they are still applied in a largely subjective fashion. Here are some of the subjective beliefs that I use in my work:

You should always prefer a simple model to a complex model if both predict the data with the same level of precision.

You should be cautious about any subgroup finding that was not pre-specified in the research protocol.

if you can find a plausible biological mechanism, that adds credibility to your results.

Advocates of Bayesian data analysis will point out that use of prior distributions will force you to explicitly state some of the subjective opinions that you bring with you to the data analysis.

Second, the use of a range of prior distributions can help resolve controversies involving conflicting beliefs. For example, an important research question is whether a research finding should “close the book” to further research. If data indicates a negative result, and this result is negative even using an optimistic prior probability, then all researchers, even those with the most optimistic hopes for the therapy, should move on. Similarly, if the data indicates a positive result, and this result is positive even using a pessimistic prior probability, then it’s time for everyone to adopt the new therapy. Now, you shouldn’t let the research agenda be held hostage by extremely optimistic or pessimistic priors, but if any reasonable prior indicates the same final result, then any reasonable person should close the book on this research area.

Third, while Bayesian data analysis allows you to incorporate subjective opinions into your prior probability, it does not require you to incorporate subjectivity. Many Bayesian data analyses use what it called a diffuse or non-informative prior distribution. This is a prior distribution that is neither optimistic nor pessimistic, but spreads the probability more or less evenly across all hypotheses.

Break #2

What you have learned
- Bayesian statistics
What’s coming next
- Prior distribution

Lay out the parameters

Blank sheet listing ECMO and control probabilities

Place half the probability on the diagonal

Probabilities of 0.005000 for every diagonal cell

Difuse prior

Probabilities of 0.00556 for every off-diagonal cell

Difuse prior

Diagonal and off-diagonal measurements combined into one matrix

Break #3

What you have learned
- Prior distribution
What’s coming next
- Likelihood

Likelihood

The second step in a Bayesian data analysis is to calculate P(E | H), the probability of the observed data under each hypothesis. If the ECMO survival rate is 90% and the conventional therapy survival rate is 60%, then the probability of observed 28 out of 29 survivors in the ECMO group is 152 out of one thousand, the probability of observing 6 out of 10 survivors in the conventional therapy group is 251 out of one thousand. The product of those two probabilities is 38,152 out of one million which we can round to 38 out of one thousand. If you’ve forgotten how to calculate probabilities like this, that’s okay. It involves the binomial distribution, and there are functions in many programs that will produce this calculation for you. In Microsoft Excel, for example, you can use the following formula.

binomdist(28,29,0.95FALSE)*binomdist(6,10,0.65,FALSE)

The table above shows the binomial probabilities under each of the 100 different hypotheses. Many of the probabilities are much smaller than one out of one thousand. The likelihood of seeing 28 survivals out of 29 babies in the ECMO survivals is very small when the hypothesized survival rate is 15%, 35%, or even 55%. Very small probabilities are represented by zeros.

Break #4

What you have learned
- Likelihood
What’s coming next
- Posterior distribution

Multiply

Product of prior and likelihood showing the need to rescale

Now multiply the prior probability of each hypothesis by the likelihood of the data under each hypothesis. For ECMO=0.9, conventional therapy=0.6, this product is 5 out of a thousand times 38 out of a thousand, which equals 190 out of a million (actually it is 173 out of a million when you don’t round the data so much). For ECMO=conventional=0.8, the product is 45 out of a thousand times 1 out of a thousand, or 45 out of a million.

This table shows the product of the prior probabilities and the likelihoods. We’re almost done, but there is one catch. These numbers do not add up to 1 (they add up to 794 out of a million), so we need to rescale them. We divide by P(E) which is defined in the Wikipedia article as

P(E) = P(E|H1) P(H1) + P(E|H2) P(H2) + …

In the example shown here, this calculation is pretty easy: add up the 121 cells to get 794 out of a million and then divide each cell by that sum. For more complex setting, this calculation requires some calculus, which should put some fear and dread into most of you. It turns out that even experts in Calculus will find it impossible to get an answer for some data analysis settings, so often Bayesian data analysis requires computer simulations at this point.

Standardize

Main diagonal of posterior probabilities

Posterior probabilities along the diagonal are small, adding up to only 6%

Upper triangle of posterior probabilities

Posterior probabilites above the diagonal are large, adding up to 94%

The probability of the upper triangle is 93.9%. That’s pretty strong evidence that ECMO is superior. There’s a lot more you can do with this data, but I just wanted to give you a small taste of how a Bayesian analysis works. First, you lay out the possible values for the population parameters. Then specify probabilities for these parameters. If you do not have a strong prior belief, or if you do not want to incorporate your prior beliefs into the analysis, spread out the probabilities in a more or less uniform fashion. Then calculate the likelihood of the data for each set of population parameters. In a survival setting, this is just a binomial distribution. Multiply the prior times the likelihood and standardize. The resulting matrix is a set of probabilities that you can sum up or manipulate in other ways.

Break #5

What you have learned
- Posterior distribution
What’s coming next
- Applications of Bayesian statistics

Applications of Bayesian statistics

Incorporate previous research
- Controls from earlier studies
Random coefficient models
Missing data
Non-standard measures
- Ranking the best

Summary

What you have learned
- Classical statistics
- Bayesian statistics
- Prior distribution
- Likelihood
- Posterior distribution
- Posterior distribution
- Applications of Bayesian statistics